1111 return stac11 items #411

jdries · 2025-06-05T13:06:34Z

TODO:

changelog entry
based on example items dict below, construct a unit test, check if it returns valid STAC 1.1

Example item entry:

{
 "5d2db643-5cc3-4b27-8ef3-11f7d203b221_2023-12-31T21:41:00Z": {
      "geometry": {
        "coordinates": [
          [
            [
              3.359808992021044,
              51.08284561357965
            ],
            [
              3.359808992021044,
              51.88641704215104
            ],
            [
              4.690166134878123,
              51.88641704215104
            ],
            [
              4.690166134878123,
              51.08284561357965
            ],
            [
              3.359808992021044,
              51.08284561357965
            ]
          ]
        ],
        "type": "Polygon"
      },
      "assets": {
        "openEO": {
          "datetime": "2023-12-31T21:41:00Z",
          "roles": [
            "data"
          ],
          "bbox": [
            3.359808992021044,
            51.08284561357965,
            4.690166134878123,
            51.88641704215104
          ],
          "geometry": {
            "coordinates": [
              [
                [
                  3.359808992021044,
                  51.08284561357965
                ],
                [
                  3.359808992021044,
                  51.88641704215104
                ],
                [
                  4.690166134878123,
                  51.88641704215104
                ],
                [
                  4.690166134878123,
                  51.08284561357965
                ],
                [
                  3.359808992021044,
                  51.08284561357965
                ]
              ]
            ],
            "type": "Polygon"
          },
          "href": "s3://openeo-data-staging-waw4-1/batch_jobs/j-250605095828442799fdde3c29b5b047/openEO_20231231T214100Z.tif",
          "nodata": "nan",
          "type": "image/tiff; application=geotiff",
          "bands": [
            {
              "name": "LST",
              "common_name": "surface_temperature",
              "aliases": [
                "LST_in:LST"
              ]
            }
          ],
          "raster:bands": [
            {
              "name": "LST",
              "statistics": {
                "valid_percent": 66.88,
                "maximum": 281.04800415039,
                "stddev": 19.598456945276,
                "minimum": 224.46798706055,
                "mean": 259.57087672984
              }
            }
          ]
        }
      },
      "id": "5d2db643-5cc3-4b27-8ef3-11f7d203b221_2023-12-31T21:41:00Z",
      "properties": {
        "datetime": "2023-12-31T21:41:00Z"
      },
      "bbox": [
        3.359808992021044,
        51.08284561357965,
        4.690166134878123,
        51.88641704215104
      ]
    }
}

Open-EO/openeo-geopyspark-driver#1111

…TAC items" This reverts commit 1109ec4. This added top-level (Collection) links to item-level links implicitly so not right conceptually. Support dedicated derived_from document only for "STAC 1.1 items" (implemented in #411). Open-EO/openeo-geopyspark-driver#1278

bossie

Did a quick review; added some points of interest inline.

I can help you with adding tests to test_views.py for these paths:

/jobs/<job_id>/results
/jobs/<job_id>/results/items11/<item_id>

and/or their signed counterparts.

bossie · 2025-09-12T14:22:39Z

openeo_driver/views.py

-                    ml_model_metadata = metadata
+
+            if len(result_metadata.items) > 0 :
+                for item_id, metadata in result_metadata.items.items():


Suggested change

for item_id, metadata in result_metadata.items.items():

for item_id in result_metadata.items.keys():

bossie · 2025-09-12T14:26:52Z

openeo_driver/views.py

+        if item_id == DriverMlModel.METADATA_FILE_NAME:
+            return _download_ml_model_metadata(job_id, item_id, user_id)


Will this ever be the case?

openeo_driver/views.py

Open-EO/openeo-geopyspark-driver#1111

bossie

Added some test skeletons to complete.

Ideally the URLs have the original /items instead of /items11.

Let me know if anything is not clear.

bossie · 2025-09-15T07:03:38Z

openeo_driver/views.py

+                    method_start = method_start + "11"
                if not signer:
-                    return url_for(".get_job_result_item", job_id=job_id, item_id=item_id, _external=True)
+                    return url_for(method_start, job_id=job_id, item_id=item_id, _external=True)


Is it possible to avoid a new endpoint (/jobs/<job_id>/results/items11/<item_id>)? I suppose the original _get_job_result_item method can also check whether the item ID corresponds to either:

an item derived from an asset (old case) or;

an actual item generated in the batch job (new case aka stac11).

tests/test_views.py

bossie

Some wrong assertions in the tests; fixing these assertions will also mean fixing the actual code.

I suggest we go over this together tomorrow.

tests/test_views.py

bossie · 2025-10-06T14:20:48Z

tests/test_views.py

+            'id': '5d2db643-5cc3-4b27-8ef3-11f7d203b221_2023-12-31T21:41:00Z',
+            'properties': {'datetime': '2023-12-31T21:41:00Z'}
+        }
+        assert res.items.get(item_id) == resp_data


Unnecessary. What matters is: the response to your item request matches the dummy item that you put in there in the first place.

bossie · 2025-10-06T14:22:39Z

tests/test_views.py

+                f"/jobs/{job_id}/results/items11/{item_id}", headers=self.AUTH_HEADER
+            )
+
+        resp_data = resp.assert_status_code(200).json


This JSON response should be a valid STAC item: at least type, stac_version and links are missing. You can find the rules here: https://github.com/radiantearth/stac-spec/blob/master/item-spec/item-spec.md

This is probably where the confusion stems from. The item that you get back from the call to get_result_metadata is incomplete so you'll have to augment it with some things (~ _get_job_result_item)

bossie · 2025-10-06T14:35:03Z

tests/test_views.py

+                            [3.359808992021044, 51.08284561357965]
+                        ]],
+                        'type': 'Polygon'},
+                    'href': 's3://openeo-data-staging-waw4-1/batch_jobs/j-250605095828442799fdde3c29b5b047/openEO_20231231T214100Z.tif',


This asset URL/URI is what the client actually receives and what he will use to read the asset from. In its current form, he can't do anything with it because he needs an endpoint as well as the proper credentials to read from this internal S3 URI.

That's why we translate this internal URI to something that the client can actually use.

Open-EO/openeo-geopyspark-driver#1111

jdries added 3 commits June 5, 2025 14:10

add field for items to metadata

488e95c

Open-EO/openeo-geopyspark-driver#1111

basics to return new-style item links if they exist

26c99fd

Open-EO/openeo-geopyspark-driver#1111

basics to return new-style item links if they exist

24a6a29

Open-EO/openeo-geopyspark-driver#1111

jdries assigned ElienVandermaesenVITO Jun 5, 2025

jdries marked this pull request as draft June 5, 2025 13:06

jdries mentioned this pull request Jun 5, 2025

#1111 return items Open-EO/openeo-geopyspark-driver#1242

Merged

bossie linked an issue Sep 11, 2025 that may be closed by this pull request

unify asset keys across STAC items Open-EO/openeo-geopyspark-driver#1111

Closed

bossie requested changes Sep 12, 2025

View reviewed changes

bossie mentioned this pull request Sep 12, 2025

split off (long) derived-from listings to separate document? Open-EO/openeo-geopyspark-driver#1278

Open

add test skeletons for results Collection and items

9398d35

Open-EO/openeo-geopyspark-driver#1111

bossie requested changes Sep 15, 2025

View reviewed changes

bossie and others added 5 commits September 15, 2025 14:12

Merge branch 'master' into 1111_return_stac11_items

67301fc

issue 1111 get_result_metadata also returns items

9374765

issue 1111 get_result_metadata also returns items

3df0a5e

issue 1111 improve exception

7aeca37

issue 1111 add signing to assets in items

eea9cb1

ElienVandermaesenVITO requested a review from bossie October 2, 2025 13:04

bossie requested changes Oct 6, 2025

View reviewed changes

bossie and others added 3 commits October 7, 2025 08:49

illustrate latest PR review remarks

dd3ba31

Open-EO/openeo-geopyspark-driver#1111

issue 1111 add mandatory properties

efd9f19

issue 1111 small change

821f0e7

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

1111 return stac11 items #411

1111 return stac11 items #411

jdries commented Jun 5, 2025

Uh oh!

bossie left a comment

Uh oh!

bossie Sep 12, 2025

Uh oh!

bossie Sep 12, 2025

Uh oh!

Uh oh!

Uh oh!

bossie left a comment

Uh oh!

bossie Sep 15, 2025

Uh oh!

Uh oh!

Uh oh!

bossie left a comment

Uh oh!

Uh oh!

bossie Oct 6, 2025

Uh oh!

bossie Oct 6, 2025

Uh oh!

bossie Oct 6, 2025

Uh oh!

Uh oh!

	for item_id, metadata in result_metadata.items.items():
	for item_id in result_metadata.items.keys():

		if item_id == DriverMlModel.METADATA_FILE_NAME:
		return _download_ml_model_metadata(job_id, item_id, user_id)

1111 return stac11 items #411

Are you sure you want to change the base?

1111 return stac11 items #411

Conversation

jdries commented Jun 5, 2025

Uh oh!

bossie left a comment

Choose a reason for hiding this comment

Uh oh!

bossie Sep 12, 2025

Choose a reason for hiding this comment

Uh oh!

bossie Sep 12, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

bossie left a comment

Choose a reason for hiding this comment

Uh oh!

bossie Sep 15, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

bossie left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

bossie Oct 6, 2025

Choose a reason for hiding this comment

Uh oh!

bossie Oct 6, 2025

Choose a reason for hiding this comment

Uh oh!

bossie Oct 6, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!